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ABSTRACT 



This document contains three papers from a symposium on 
research methods in human resource development (HRD) that was conducted as 
part of a conference on HRD. "Utilizing Electronic Mail To Survey Human 
Resource Development Practitioners: A Comparison between Electronic Mail and 
the U.S. Postal Service for the Purpose of Data Collection: Reducing the 
Costs of Bureaucracy" (Harold Shoemaker, James J. Kirk) reports on a study 
establishing that electronic mail elicited a significantly higher response 
rate than regular mail (41% versus 25%) but did not alter response patterns 
by gender or years in the HRD profession. "Trends in the Literature: A 
Comparative Analysis of 1998 HRD Research" (Loretta L. Donovan, Victoria J. 
Marsick) documents the following three trends in HRD research: (1) HRD has 

made strong inroads as an area of professional practice; (2) the field 
continues to use qualitative and quantitative tools relatively equally; and 
(3) the number of articles published in the field increased by 50% this past 
year. "Ref lection-in-Action of a Research Partnership" (Wendy E. A. Ruona, 
Darren C. Short) outlines the learning points identified as key to successful 
research partnership projects by two researchers who spent more than 2 years 
in such a partnership. The papers contain reference sections. (MN) 
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Utilizing Electronic Mail to Survey Human Resource Development Practitioners: A 
Comparison Between Electronic Mail and the U.S. Postal Service for the Purpose of 
Data Collection: Reducing the Costs of Bureaucracy 



Harold Shoemaker 

University of Southern Mississippi Gulf Park 
James J. Kirk 

Western Carolina University 



Electronic mail (E-mail) has enabled researchers to collect data systematically from any 
location in the world in seconds, and charges for online services have been less than for 
postage and free to educational users. E-mail was utilized to solicit and collect data as part 
of a study of transfer of training strategies among industry trainers and supervisors. The 
findings hold implications for HRD researchers using E-Mail to collect data. 



Keywords: E-Mail, Data, Collection 

The utilization of the U.S. Postal Service (regular mail) has been more costly than the utilizatioof E-mail for data 
collection. Preparing a document for regular mail has been more difficult to edit and analyze, change, and sort than 
preparing a document for E-mail. Regular mail has alsotaken longer to deliver than E-mail. According toThach 
(1995) response rates to surveys have been considerably lower using regular mail than with E-mail. Other 
problems found in using regular mail have included less honesty of responses and slower responses than E-mail. 



Utilization of E-Mail 

E-mail has allowed users with a computer, a modem, and an on-line service to transmit and receive information 
from virtually anyplace in the world $teinfield, 1990;Zuboff, 1988). When utilizing E-mail for survey research, 
the user has been able to collect data systematically by composing questionnaires and delivering them to an online 
sample of the population. Respondents could then receive, complete, and return the questionnaire via E-mail. 
Research utilizing E-mail for data collection has been projected to increase because of the growth of online services 
throughout the world (Thach, 1995). 

Strategies for Using E-Mail 

E-mail has had four important features that make it useful for survey research: 

1. 1 .Messages could be transmitted to any location in the world in seconds (depending on the scope of the 
network). 

2. Messages could be sent, read, and replied to at the convenience of the user. Participants could take their 
time to think about their response and answer when they were ready. 

3. The intended receiver has normally been the only person who read the messages. Typically, there have 
been no secretaries or office personnel opening or sorting mail (there has been some change in larger 
organizations, however, where executives received over 100 E-mails per day and secretaries have been 
required to sort them). However, E-mails have had a better chance of being opened and read by the 
intended receiver than traditional mail might §proull, 1986). 

4. E-mail messages have not required a hard copy, adding an ephemeral quality to E-mail that could not be 
duplicated by traditional mail. E-mail messages could be saved and printed to a hard copy at the discretion 
of the receiver (Sproull, 1986). 
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Limitations in Use of E-mail 



The use of E-mail has been limited to populations who had access to the equipment required and had the 
knowledge to use the equipment. However, as technology improved and equipment costs and online fees have 
fallen, more diverse populations have had access to E-mail {Tiach, 1995). Utilizing E-mail for research has had 
some potential problems, however. Data could be lost due to software or hardware problems. To overcome these 
types of problems, it has been considered prudent to make back-up copies of filesSJynodinos & Brennan, 1988). 
As illustrated in Figure l,Thach (1995) listed the advantages and disadvantages of using E-mail for mail surveys. 

Technology has increased user expectations of format issues such as color, spacing, and location of items. 
These issues, on the other hand, were more easily addressed because of the possibility of sending colored surveys 
complete with graphics to users on networks for less than the cost to reproduce and mail colored copies of paper 
surveys. Technology has created more potential for development of effective questionnaire layouts and 
presentations in the future (Thach, 1995). 



Advantages 


Disadvantages 


Cost-savings— less expensive to send questionnaires 
over online network than to pay postage for paper 
questionnaires or interviewers salaries. 


Sample demographic limitations— population and sample 
are limited to those with access to a computer and online 
network. 


Ease of editing^nalysis— simpler to make changes 
to questionnaire after pretesting and easier to copy 
and sort data, since itdoesn’t have to be re-typed. 


Lower levels ofconfidentiality-due to the open nature of 
most online networks, it is difficult to guarantee anonymity 
and confidentiality. 


Faster transmission time — questionnaires can be 
delivered to recipient in virtually seconds, rather 
than days as with traditional mail. 


Layout and presentationissues— constructing the format of a 
computer questionnaire can be more difficult the first few 
times, due to lack of experience for some researchers. 


Easy use ofpreletters (invitations) — invitations to 
participate can be sent and responded to in a very 
short time, thus providing the researcher with an 
estimate of the participation level. 


Additional orientationinstructions— extra instructions and 
even orientation to the computer and online system may be 
necessary in order for respondents to complete the 
questionnaire online. 


Higher response rate— research shows that response 
rates on private networks are higher with electronic 
surveys than with paper surveys. 


Potential technical problems with hardware and software 


More candid responses— research shows that 
respondents will answer more honestly with 
electronic surveys than with paper surveys or in 
interviews. 

Potentially quicker response time with wider 
magnitude ofcoverage— due to the speed of online 
networks, participants can answer in virtually 
minutes or hours, and coverage can be global. 





Figure 1 Advantages and Disadvantages of Electronic Mail Surveys 

Thach, L. (1995). Using electronic mail to conduct survey research Educational Technology. 35 (2L 31. 



Pretests 



Pretests (pilot or field tests) could be sent directly to a small pilot group of users at virtually no cost and returned 
with suggested revisions in the same manner. Editing and revisions were also quicker than with hard copy, because the 
questionnaire was already developed using a computer format. Any changes could be made directly on the screeiK^esler 
& Sproull, 1986; Synodinos & Brennan, 1988). 



Confidentiality 

Confidentiality has been an important issue in using E-mail surveys. Because electronic networks have been open 
in nature, it has been more difficult to guarantee anonymity or confidentiality to respondents than with regular mail surveys 
(Price, 1975). Most online services sent the E-mailrespondents' E-mail address along with the survey response, allowing 
the researcher to know who responded and how. Although this eliminated anonymity, it did not preclude confidentiality 
(Thach, 1995). In an anonymous questionnaire, no one knew the name of the respondent, not even the researcher. 
Confidentiality, however, has implied a promise that the researcher would keep the name and responses of the respondent 
confidential (Berdoe et al., 1986). 

Cost-Saving 

When surveys have been implemented online, postage fees have been avoided. Charges for online services have 
been less than for postage, and educational users of the Internet have not been charged for personal use timdljiach, 1995). 
Kiesler and Sproull (1986) reported that the cost-savings benefits of using E-mail for researchwill be one of the major 
reasons for the growth of E-mail in the future. 

Response Findings 

Response rates for E-mail have generally been found to be positiveXhach, 1995). Sproull (1986) found response 
rates from E-mail surveys to be 20% higher than hard-copy mail questionnaires. 

Kiesler and Sproull (1986) found that E-mail respondents answered more items and made fewer mistakes on the 
questionnaire than their typical mail counterpart. In a later study, Walsh et al. (1992) achieved a 76% response rate to their 
E-mail survey using a random sample. That improved to a 96% response rate when a self-selecting group was surveyed. 
Online questionnaires have proved to improve speed of transmission. Rather than waiting for surveys to arrive by traditional 
mail, the online questionnaire could be delivered in virtually seconds^proull, 1986; Synodinos & Brennan, 1988). 

Candid Responses 

Kiesler and Sproull (1986) found that respondents tended to answer more honestly and with fewer social inhibitions 
than did comparison groups who answered paper questionnaires and/or face-to-face interviews. It was suggested that this 
type of candid responseis due to the shielding of social context of traditional communication. Many users have had the option 
of using alias E-mail addressnames which allowed them to disguise their identity. Although these same attributes were true 
of paper questionnaires, E-mail elicited morecandor (Sproull, 1986; Synodinos & Brennan, 1988). 

Speed and Magnitude 

Speed and magnitude of coverage of response rates have been much better than traditional paper questionnaires 
and interviews largely because of the large number of people on private online networks and the ability to send a large 
number of questionnaires out very quickly fhach, 1995). The magnitude of E-mail has been growing to the point where 
many networks could reach remote locations around theworld and receive responses in seconds (Walsh et al., 1992). 
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Research Questions 



1 . Is there a significant difference between results from surveys sent b£-Mail and surveys sent by regular mail? 

2. Is there a significant difference between results from surveys sent bg-Mail and surveys sent by the U.S. Postal Service 

with respect to the gender of the respondent? 

3. Is there a significant difference between results from surveys sent bg-Mail and surveys sent by the U.S. Postal Service 

with respect to the experience as a human resource development professional of the respondent? 



Methodology 

Based on the findings in the literature, E-mail was utilized to solicit and collect data as part of a study of transfer of training 
strategies among industry trainers and supervisors who were members of the American Society for Training and 
Development (ASTD). 

The ASTD was chosen for this study of trainers and supervisors because of its claim of a membership of over 
70,000 professionals in the area of training and development throughout the United States and other countries. ASTD has 
supported (a) professional growth and learning, (b) leadership skill development, (c) networking opportunities, (d) useful 
current information, and (e) training career opportunities for trainers, supervisors, and others in related professions. 



Sampling Method 

The sample used in this study was selected from a randomized list of trainers andupervisors provided by ASTD 
during the summer of 1998.Krejcie^s and Morgan’s (1970) Table for determining sample size from a given population was 
used to determine the sample size of 383 trainers and supervisors. 



Response Rate 

Out of the 383 subjects (383 original subjects less 35 subjects who stated that they were not interested in the study), 

1 18 were completed and returned. Of the 1 18 returns, 7 were unusable, leaving 1 1 1 usable responses, yielding an overall 
29% usable response rate. 

Prior to the actual survey a pre-survey letter requesting participation was sent. From that request, 92 respondents 
indicated interest in taking part in the study, 35 respondents indicated that they were not interested in taking part in the study, 
and 205 were returned as undeliverable. Fifty-one subjects received the pre-survey letter but did not respond. 

Of the 348 subjects remaining, the survey was sent via E-mail to 143 subjects (92 who had agreed to participate 
and 51 who did not respond), and 205 were sent via regular mail to the undeliverable E-mail group. Of the responses 59 
were returned from the E-mail surveys (41% response rate) and 52 were returned from regular mail (25% response rate). 



Results and Findings 
Research Question One 

\ 

Research question number one: Is there a significant difference between results from surveys sent bp-Mail and 
surveys sent by regular mail? 

Total responses from each respondent group $>Mail group and regular mail group) were compared to the non- 
respondents. A Chi Square was used to compare the respondents from thdi-Mail surveys, the respondents from regular mail 
surveys, the non-respondents from theE-Mail surveys, and the non-respondents from regular mail surveys. Table 1 displays 
the actual response rates for E-Mail returned, not returned and total and for regular mail returned, not returned and total. 
The respondents from theE-Mail surveys were found to be significantly greatethan the respondents from regular mail (2, 
205) X 2 = 9.795, p = .002. 

Table 1 

Rates of Return fromE-Mail and Regular mail Surveys 




40-1 





Returned 


Not Returned 


Total 




E-Mail 


59 


84 




143 


Regular Mail 


52 


153 




205 


Totals 


111 


237 




348 



X 2 = 9.795, N= 348, df = 1, p = .002 



Research Question Two 

Research question number two: Is there a significant difference between results from surveys sent b^-Mail and 
surveys sent by the U.S. Postal Service with respect to the gender of the respondent? 

Mean scores for each respondent group and combined mean scores were used to determine any differences in 
responses with respect to the gender of the respondent. AChi-Square was used to compare the mean scores of the 
respondents who indicated that they were female to the respondents who indicated that they were male. There was no 
significant difference between the responses of the females and the males. F (1,107) p = .075. 



Research Question Three 

Research question number three: Is there a significant difference between results from surveys sent bp-Mail and 
surveys sent by the U.S. Postal Service with respect to the experience as a human resource development professional of the 
respondent? 

Mean scores for each respondent group and combined mean scores were used to determine any differences in 
responses with respect to the experience as a human resource development professional of the respondent. jChi-Square 
was used to compare the mean scores of the respondents among the four experience groups (less than 3 years, 4 to 8 years, 
9 to 15 years, and over 15 years). There was no significant difference between the responses of the four groups. F (3,107) 
p = .875. 



Conclusions 

The higher rate of response from the overall E-mail surveys (41% compared to 25% from regular mail surveys [a 64% 
improvement]) heavily exceedsSproull’s (1986) findings that response rates from E-mail surveys were 20% higher than 
hard-copy mail questionnaires. 

However, the findings did not support Walsh efel’s (1993) findings of 76% and 96% response rates from E-mail 
surveys. As the literature suggested, when surveys have been implemented online, postage fees have been avoided, charges 
for online services have been less than for postage, and educational users of the Internet have not been charged for personal 
use time (Thach, 1995). Kiesler andSproull (1986) reported that the cost-savings benefits of using E-mail for researchvill 
be one of the major reasons for the growth of E-mail in the future. 



Recommendations 

Based on the findings of this study and from the studies cffproull (1986) and Brennan (1988), E-mail data collection results 
would yield response rates similar to those of Walsh et al (1993) by (a) sending an E-mail letter to all subjects in the 
population who have E-Mail addresses asking them if they are interesting in participating, and (b) randomly selecting the 
desired sample size from those who agreed to participate. 




7 



40-1 



New Knowledge for Human Resource Development Researchers 

Utilizing E-mail for data collection, HRD researchers would be able to (a) improve response rates, (b) collect data in a timely 
manner, (c) collect data at a lower cost than other methods. Other benefits discussed in the literature review should also be 
considered. 

HRD researchers using E-Mail to collect data are not restricted to the same limitations of sample size that might 
be experienced using regular mail. UtilizingE-Mail to conduct surveys saves time in preparing multiple mailings collecting 
data, and conducting follow-up mailings; saves paper and envelopes to enable hard copies of mailing materials; and saves 
postage. 
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Trends in the Literature: A Comparative Analysis of 1998 HRD Research 



Loretta L. Donovan, M.A. T. 

Victoria J. Marsick, Ph.D. 

Teachers College, Columbia University 



Contributions of the literature of the field are significant indicators of trends in the profession. Evidence 
of the direction of the field is presented in a qualitative analysis of the HRD articles published in 1998. 
The types of research articles and the sources of studies are compared similarly to the previous work of 
Hixon and McClernon (1999), and are classified by research journals, research methodologies, 
analytical and statistical tools, and organizational participants. 



Keywords: HRD Research, HRD Research Trends, HRD Literature 

This analysis is focused on journal articles that report research meaningful to both the theorist and the workplace 
learning professional, a category that has appeal to scholar-practitioners ( Ruona, 1998; Leimbach, 1997; McClean, 
1997; Watkins, 1994). The review extends the earlier analysis by Hixon and McClernon (1999) of research articles 
considered for a major award given by a professional association in the field of HRD. The review demonstrates the 
continuing increase in the rate of publication for HRD articles across a broad range of journals in related fields. 
Following a discussion of context and the rationale for such analyses, we describe methods used for this analysis 
and strategies employed in identification of articles to be included in the pool of research articles considered for the 
award. We then present and interpret our findings. 



Context and Rationale 

Human Resource Development (HRD) has established itself as a professional field with its own theory base, 
research and practice. It is "... a process of developing and/or unleashing human expertise through organization 
development (OD) and personnel training and development (T&D) for the purpose of improving performance" 
(Swanson, 1998). The literature of the field is a meaningful gauge of trends in this profession which has comprised 
the development of the individual, the group and the organization with a focus on relevant business results. HRD 
research is often published across a broad range of articles in journals associated with a variety of disciplines. An 
inventory of journals that published HRD research articles grew from twelve for the period 1990-1994 to twenty- 
two as identified by Sleezer and Sleezer in 1997, and reviewed by Hixon and McClernon in 1999. This indicator 
alone demonstrates an increase in the attention to the field by refereed research journals. 

Scholars in the field have also paid attention to systematizing the process of review. By building on the 
work of Arnold (1996) and the Sleezers (1997), Hixon and McClernon provided a more consistent picture of the 
emerging field. Those authors categorized not only the research journals, but also the percentages of research 
articles by type, and the number of studies by participant source. They were therefore able to conclude that more 
HRD research was published in 1997; that a variety of journals published the articles; and that the relative types of 
research differed from those reported in previous studies (although not necessarily a trend). Their research also 
suggested to them that a more concise and comprehensive definition of HRD would result in the identification of 
literature that is more relevant to the field. The work of identifying and classifying the literature is continued by the 
present study. A comparative analysis of the HRD research literature published in 1998 has been analyzed and is 
reported here using the framework developed by previous authors. 
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Purpose and Methods 

The purpose of this study is to review and report trends in HR D research literature that is interest to scholar- 
practitioners and that has been published in scholarly journals in 1998. Donavan screened and collected articles as 
the first phase of a search for research articles in HRD to be considered for a prestigious professional association 
award. Content analysis was used to determine similarities and differences to the findings of Sleezer and Sleezer 
(1997) and Hixon and McClemon (1999) as to: the research journals, types of research articles, the sources of 
studies, research methodologies, analytical and statistical tools, and organizational participants represented by the 
articles published. 

The researchers employed content analysis to develop an integrative view of the trends in the HRD 
research articles published in 1998 that were considered for this award, and to then compare the results with 
previously published studies of similar HRD research. To achieve this goal, the literature was inventoried, and 
research articles were surveyed and categorized. Data collected for each article included in the review were: the 
research journal in which it was published, the institutional source of the study, the research methodology and tools 
applied, and the keywords used to describe the article. The resulting data have been analyzed and interpreted. 



Inventory of the Literature 

In order to systematically compare the 1998 HRD research to studies collected and analyzed in 1997 and prior 
years, the literature was selected for review using the criteria established by Hixon and McClemon (1999): 

(1) The article was published in a refereed journal in [1998] ; 

(2) The article addressed the practical implications of some issue of relevance to the practice of workplace 
learning and performance ; 

(3) The article was published in English ; 

(4) The article reflected systematic research; 

(5) The article fit with Swanson's (1998a) definition [of HRD]. 

The process of identifying the research articles began with ABI/Inform, ERIC, and PsychLit, the same sources used 
by the Sleezers (1997), and Hixon and McClemon (1998), to review the journals previously surveyed. The abstracts 
and articles were matched to the criteria listed above. Additionally, Donavan used ProQuest and Article First to 
locate articles in journals not formerly referenced. The tables of contents of copies of journals suggested by 
members of the community of HRD academics and practitioners were manually examined as well. 

Keywords that were used to search the databases came from HRD, OD, and T&D. The terms included: 
change management, employee development, human resource, human resource development, learning, 
management development, organizational behavior, organizational change, organizational learning, performance, 
research, return on investment, studies, training and development, transition, workplace learning. 

A total of 101 potential articles were identified in 21 journals from all database and manual searches 
conducted. For those journals not previously known to be refereed, ProQuest was used to determine that criterion. 

In some instances, it was necessary to read the contributor guidelines to confirm that the journal was refereed. 

When Swanson's definition was applied, a total of 97 research articles in 15 journals proved to be relevant. 

Categorizing the Articles 

As each article was read, Donovan classified the article by journal title, the participant source, the 
research methodology and tools used, and the subject of the research. As had been previously done, the list of 
journals were compared to previous ones, first to Hixon and McClemon (1999) and then to Sleezer and Sleezer 
(1997). The types of research were matched to Arnold's (1996) typology. The industry or participant group was 
classified along similar lines to those used by Hixon and McClemon (1999). A new category, the occurrence of 
certain keywords to describe the articles in particular journals, was added in this study. The process of 
categorization did not involve other criteria such as quality of the research, length of the article, or whom the 
researchers were. 
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Findings 



Analysis of findings yielded a comparison of current and previous journals reviewed and a list showing the 
distribution of research articles according to Arnold fc (1996) typology and compared to Arnold fc (1996), Sleezer 
and Sleezerfc (1997) and Hixon and McClemonfc (1999) data. The analysis highlights the number of studies by 
industry or participant source, similarly compared to data from the studies completed by Arnold (1996) and by 
Hixon and McClemon (199). In addition, a new analysis has been done of the areas of focus of the research. 



Table 1. Comparison of HRD Research Articles in 1997 and 1998 Study Data Bases 



Journal 


Number of HRD 
Research Articles 
Published in 1998 


Number of HRD 
Research Articles 
Published in 1997 


Adult Education Quarterly** 


6 




Academy of Management Journal ++ 




i 


The Academy of Management Review 


0 


i 


Applied Psychology: An International Review ++ 




i 


Educational Psychology ++ 




i 


European Journal of Operational Research** 


1 




Group & Organization Management 


0 


4 


Human Relations 


11 


4 


Human Resource Development Quarterly 


16 


7 


Human Resource Management 


4 


1 


Industrial Relations 


0 


3 


International Studies of Management & 
Organization** 


4 




Journal of Applied Psychology 


5 


4 


Journal of Applied Behavioral Science 


7 


5 


Journal of European Industrial Training ++ 




1 


Journal of Management** 


0 




Journal of Organizational Behavior 


8 


7 


Journal of Vocational Behavior 


0 


5 


Management Learning** 


4 




Management Science** 


3 




National Productivity Review ++ 




1 


Organization Development Journal ++ 


0 


4 


Organizational Dynamics 


8 


1 


Organization Studies 


5 


3 


Performance Improvement Quarterly 


10 


4 


Personnel Psychology 


5 


3 


Public Administration Quarterly ++ 




2 


Public Administration Review** 


0 




Public Personnel Management ++ 




2 


Training Research Journal ++ 




1 


TOTAL ARTICLES 
1 Hixon & McClemon (1999) 

**Not included in a previous study. 
++Not included in this study. 


97 


66 



Data analysis has resulted in a report of the distribution of HRD research articles by journal as shown in 
Table 1, which includes both the 1997 and 1998 data. The journals that published HRD research in 1998 included 
Human Resource Development Quarterly. Human Relations . Performance Improvement Quarterly . Journal of 
Organizational Behavior. Organizational Dynamics . Journal of Applied Behavioral Science . Adult Education 
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Quarterly Journal of Applied Psychology . Organization Studies . Personnel Psychology The number of articles 
published in each journal included (or omitted) from the 1999 and current studies is shown. 

We repeat the caution here of Hixon and McClemon ( 1 999) who noted that their study was not an exact 
replication of those previously conducted. This study, as well, differs from the others. It has used the same criteria 
for article selection, but the journals included in the study are not identical to those reviewed in either the 1997 
study by the Sleezers or the 1999 work of Hixon and McClemon. As can be seen in Table 1, additional new 
journals in this review focused on adult and management learning, as well as on international studies. However, 
some journals not included were also international; several journals in 1997 that were not included here focused on 
public administration. 

Type of Research 

Each of the articles identified was categorized according to the typology of Arnold (1996). He 
characterized research using four classes: field or lab experiments, descriptive case of field studies, library 
research/speculative, and theoretical model or instrument construction. Among the 1998 articles analyzed, the 
distribution among these classes were: 1 1 field or lab experiments; 58 descriptive case or field studies; 15 library 
research/speculative studies; and 13 works on theoretical model or instrument construction. 

A comparison of the occurrence of each type of research in Arnold's study (1996), Hixon & McClemon's 
study (1999), and the current study are shown in Table 2. Of the 1998 articles reviewed, 11% of the articles were 
field or lab experiments, as compared to 7% in 1996 and 20% in 1997; 60% were descriptive case or field studies, 
as compared to 53% in 1996 and 65% in 1997; 16% were library research or speculative, as compared to 35% in 
1996 and 12% in 1997; and 13% were theoretical model or instrument construction, as compared to 5% in 1996 
and 3% in 1997. The differences noted in the numbers of studies in each category to the previous year are not as 
marked as those found by Hixon & McClemon (1999). Experiments seem to be moving downward; and there are 
slight increases or decreases for field studies and library / speculative research. The largest increase is in 
theoretical model or instrument construction. As has been stated before, the selection of journals and the bias of the 
reviewers may account for the magnitude of these increases or decreases. 

Table 2. Percentages of Research Articles by Type in 1996, 1997, and 1998 Study Data Bases 



Research Type 


Articles in Arnold’s 
Study 1 


Articles in Hixon & 
McClemon’s Study 2 


Articles in 1998 


Field or Lab Experiments 


7% 


20% 


n% 


Descriptive Case or Field Studies 


53% 


65% 


60% 


Library Research/Speculative 


35% 


12% 


16% 


Theoretical Model or Instrument Construction 


5% 


3% 


13% 


TOTAL 


100% 


100% 


100% 


'Arnold (1996) p. 819 








2 Hixon & McClemon (1999) p. 900 








Qualitative vs. Quantitative Research 









The next analysis of the research studies determined whether the type of data, methods and tools were 
oriented to qualitative or quantitative research. A total of 46 articles reported using qualitative measures; that 
represented 47% of the studies. Quantitative measures were used by 5 1 of the studies; or 52% of the researchers. 
This is somewhat of a decrease in quantitative studies from Hixon & McClemon (1999) who had found 68% of the 
studies in 1997 to be quantitative and 32% to be qualitative. The vast majority of the current qualitative research 
were primarily interview studies and case studies. This may reflect an increased intellectual interest in 
constructivism and post-modernism which pay close attention to context and to the meaning that people make of 
their unique situations. 
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The quantitative studies were categorized using Arnolds (1996) list of methods: factor analysis, Chi 
square, ANOVA, correlation, means, proportions, variance and none. Of the quantitative studies reviewed most 
relied on simple measures such as means and variance. Factor analysis was used by 10 studies; ANOVA by 1 1; 
correlation by 25; and complex measures by 4. 

Research Settings and Focus 

Research setting and focus of articles reviewed in 1998 could, in part, be influenced by the selection of 
journals for review. Table 3 shows a comparison of number and percentage of studies by setting. A total of 59.8% 
of the articles reviewed in 1998 reported on studies conducted in business and industry, which is a large increase 
from Hixon and McClemonfc (1999) review of 1997 research. Mixed organization types (combinations of 
business, education, government agencies) were used for 10.3% of the studies. Higher education accounted for 
7.2% of the research reported. A total of 7.2% of the studies were done in governmental agencies. Another 7.2% of 
the articles reported no subject audience. The remaining 8% of the research articles included studies conducted in 
professional services organizations, healthcare settings, non-profits, and the community. 

Table 3. Comparison of Number of Studies by Setting 



Setting 


Articles in 1998 


Articles in 1997 




Number 


Percent 


Number 


Percent 


Business/Industry 


58 


59.8% 


22 


33.3% 


Mixed Organizations 


10 


10.3% 


5 


7.8% 


Education 


7 


7.2% 


9 


13.6% 


Government 


7 


7.2% 


9 


13.6% 


Unknown 


7 


7.2% 


8 


12.1% 


Professionals 


3 


3.1% 


2 


3% 


Healthcare 


2 


2.1% 


2 


3% 


Non-profits 


2 


2.1% 


0 




Community 


1 


1.0% 


0 




Military 


0 




3 


4.5% 


Financial 


0 




4 


6.1% 


Proprietary Education 


0 




2 


3% 


TOTAL 


97 


100% 


66 


100% 



1 Hixon & McClemon (1999) 

The research reported in 1998 was conducted in a variety of countries although the overwhelming majority (about 
85%) of articles focused on studies completed in the United States. The remaining 15% of the articles reported 
research from the following regions of the world (in order of frequency): Western Europe, primarily the UK; 
Eastern Europe, including Hungary, Poland and Russia; Canada; the Far East, including China and Japan; and 
Northern Europe. 

The research studies reviewed in 1998 focused largely on employees across the organization, from senior 
management and entrepreneurs to new hires. Table 4 articles shows the distribution of articles by area of focus 
using the following categories: learning theory and practice, performance, managerial behavior, change, diversity, 
business development, leadership development and practice, learning organization, technology, organizational 
culture, human resource development, instructional design, return on investment, and knowledge management. 
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Table 4. Percentages of Research Articles by Focus 



Area of Focus 


% of Articles 


Learning Theory and Practice 


22% 


Performance 


14% 


Managerial Behavior 


11% 


Change 


9% 


Diversity 


9% 


Business Development 


8% 


Leadership Development and Practice 


8% 


Learning Organization 


5% 


Technology 


4% 


Organizational Culture 


3% 


Human Resource Development 


2% 


Instructional Design 


2% 


Return On Investment 


2% 


Knowledge Management 


1% 


TOTAL 


100% 



Discussion 

The status of HRD research has been reviewed in this study. With the insights gleaned from this review, our 
comments will fall into two broad areas the state of HRD research as reviewed here and the limitations and 
difficulties of a study such as this one. 

It seems that the amount of published research continues to grow, and that the number of journals that 
include HRD research have expanded. One cannot infer too much regarding trends from such a limited data set, 
especially given the differences between the data base of journals for 1997 and 1998. A few clear trends seem to 
stand out, most notable among them being the increase in number of studies in business and industry; and the 
paucity of research in English outside of the United States. 

However, numbers alone do not tell the story of research in Human Resource Development. There is some 
variance in the nature and focus of the studies conducted, although it is difficult to say why these differences exist 
or to be sure that they are not due to the nature of the sample. On the other hand, studies of interest to scholar 
practitioners are bound to be sensitive to rapid environmental changes that might explain an interest in 
contextualized studies that provide descriptive analyses of forces within a given industry, sector, or employee base. 
Such exploratory research may eventually give rise to cross-case analysis. Additionally, postmodernism calls into 
question the very possibility of conducting large-scale causal studies with high levels of cross-context explanatory 
power. 

Although findings with respect to setting and focus may well be influenced by the choice of journals, this 
analysis reports an increase in research with a focus on business and industry along with continued interest in 
studies in other settings. A decrease in the number of journals reviewed in the area of public administration may 
well account for the drop in studies considered here in the public and not-for-profit sector. One can speculate that 
the increase of interest in HRD in business and industry is caused by positive factors, such as an increase in 
understanding of the field of HRD and its potential for positive impact on business results; or the increasing 
influence of HRD practitioners, consulting both internally and externally; or the availability of resources for 
research given the strong economy. It is also possible that funding for HRD research is not as available in the other 
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sectors. Or that the number of practitioners working in those sectors is somewhat lower. Nevertheless, there are 
practical implications in many sectors and settings related to issues significant to the practice of workplace 
learning and performance. Is it possible that scholars in HR D are focusing more attention on sectors that promise 
more lucrative work? On the other hand, are there other reasons for the distribution of research studies? If so, 
what are they? 

Continuing challenges to literature review in the field of Human Resource Development have become 
evident to the researchers involved in the current and previous studies. From our vantage point they can be grouped 
in three realms. The first of these is related to publication and availability of refereed journals. The second results 
from inadequate abstracts and the wide variance in the selection of keywords that identify articles. The final 
challenge is the experience and professional interests of the researcher. 

Any comparative literature search of HRD articles to survey the work of the field is affected by the 
accessibility of journals and the availability of articles. The issues that appear to impact access are continued 
publication of the journal and the role of academic libraries in providing online or paper subscriptions. Several 
journals included in the 1997 and 1998 studies did not appear in library databases as this research was conducted. 
Individual academic libraries may subscribe to a limited number of publications. Contractual agreements prevent 
interlibrary loan of more than a single article from one issue of a journal. 

The online research resources include, at the disposition of publishers, a citation, and/or abstract and/or 
full-text for an article. Authoring an abstract is an inexact process at best. In the early stages of this research, 
abstracts of more than 50 articles, for which full-text was not available online, were earmarked for possible 
inclusion in the study. Fewer than ten of those abstracts represented articles that met the criteria established. Most 
often the abstract did not differentiate a field or descriptive study from a theoretical article; or a position paper 
from model building. Literature studies were not distinguished from those involving human subjects. These 
shortcomings can easily result in the omission of articles when initial selection depends upon an abstract. 

Researcher experience and professional interests are limitations that affect literature review as well. 
Special interests, previous research and practice often expose HRD scholars to certain segments of the field. In 
doing so, they are prone to recognize certain terminology, be aware of emerging trends, and examine particular 
literature. At the same time, other terms, trends and publications may lie outside of their practice. The selection of 
articles for inclusion in a study such as this is thus necessarily subjective. 



Conclusions 

Human Resource Development has made strong inroads as an area of professional practice. The field continues to 
use both qualitative and quantitative tools in a relatively equal balance. An increase of 50% in the number of 
published articles in the field occurred in one year. As the balance of interests and sources of research continue to 
evolve, we support continued process of self-examination. We must know more about the sponsors and venues that 
make research possible, and about the journals that are willing to publish it. We encourage future researchers to 
study the types of research being done, and to analyze those data in both comparative and in new ways. Both 
annual and five-year reviews would be beneficial to members of the field of Human Resource Development. 
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Reflection-in-Action of a Research Partnership 
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In an applied field like HRD it is vital that practioners be active participators in the research enterprise. 

To ensure that participation, HRD professionals must continue to explore and address the multitude of 
challenges inherent in research/practice collaborations. This paper reports two researchers reflection-in- 
action during a 2+ year research partnership , and outlines learning points they have identified as key to 
successful research partnership projects. 

Keywords: Partnership research, Theory -to-practice 

It is commonly agreed upon that Human Resource Development (HRD) is an applied field driven by its practice. In 
recent years, there has been much lament about the disconnect between theory and practice, yet this gap only seems 
to be widening as the pace of organizations today steadily increases and theory continues to be labeled as invaluable 
for the practical problems that organizations face. Although this problem is extraordinarily multi-faceted, a certain 
contributing factor is the extent to which academics and practioners partner together to conduct HRD research. 

Research and practice must be more strongly linked to ensure a solid foundation on which the field can 
build and be more successful in achieving its goals. Modem views of the relationship between theory and practice 
have helped to downplay, and many would argue entirely remove, the superiority of theory to practice. Rather, 
practice is and should be viewed as the starting and ending place for research. Carr (1980) urges us to no longer 
view practioners as objects for theoretical inspection or as clients who accept and apply theoretical solutions” (p. 
67), but rather to recognize that it is their “. .. active participation in the theoretical enterprise that is an indispensable 
necessity” (p. 67). Carr and Kemmis (1986) urge us to take a praxiological view of theory and practice — they are 
dialectically related and ‘to be understood as mutually constitutive, as a process of interaction which is a continual 
reconstruction of thought and action” (p. 34). In order to gamer the active participation of HRD practioners in 
building and then using HRD research/theory, HRD professionals (both academics and practioners) must make great 
strides in improving the design and implementation of successful research-practice partnerships. 

It is quite possible that the fields professionals need new models on which to build a solid research and 
practice partnership. One of the major contributors doing work to lessen the gap between research and practice has 
been Jacobs (1996, 1997, 1999) who is developing a model for partnership research, which he defines as the 
‘process of improving HRD practice through research” (Jacobs, 1999, p. 874). With each revision of this evolving 
model HRD professionals are offered more specific advice on how to collaborate using a different model than that 
which has been commonly accepted and employed in past HRD research. The real emphasis of this model thus far 
has been on collaboratively defining a research study out of problems found in organizations. 

Less work has been done around exploring the multitude of challenges inherent in research/practice 
collaborations. Leimbachfc and McLeanfc rousing 1997 Academy of Human Resource Development (AHRD) town 
forum provided some insight into perceived challenges through both the eyes of the practitioner and the academic. 
Bassi (1996) also contributed in this vein in identifying motivations for partnering as well as “Deal Makers” and 
“Deal breakers” (p. 709). Nimtz, Coscarelli, and Blair (1996), Jacobs and Moore ( 1998 ) and Kehrhahn and Verrilli 
(1999) have all recently offered systematic case studies of corporate-university partnerships. 

While a handful of other studies of HRD-specific research partnerships exist, and even many more within 
the broader social sciences, there is an on-going need to continue to surface and share critical reflections and lessons 
being learned by research partners. Bassi (1996) stated, ‘before developing a set of principles for guiding 
partnership research, it is useful to delineate the perils and opportunities inherent in partnerships” (p. 707). As a 
field, HRD must continue to expand its understanding of the unique challenges being faced by HRD researchers and 
practitioners attempting to partner to address organizational problems as well as to produce valuable research. 
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Purpose of this Study 

The purpose of this study is to contribute towards the effort of understanding more about research and practice 
partnerships in HRD. This paper will describe and document the course of a 2+ year research collaboration, 
exploring challenges faced, what was achieved, and key lessons learned from both the perspectives of the academic 
and the practitioner. 



Methodology 

The researchers, one a University professor and the other the Head of HRD Consultancy for a large British public- 
sector organization, began a partnership to address an emerging organizational problem as well as to conduct HRD 
research beginning first quarter, 1998. Parallel to this research, the authors studied the partnership itself. Each of 
these efforts is briefly described below. 

The Research Project 

The authors collaborated as part of an effort to contribute to the validation of an instrument entitled The 
Learning Transfer System Inventory (LTSI) (Holton, Bates, Ruona, Leimbach, 1998). During May to September 
1998, the LTSI was administered in 40 non-residential, non-mandatory training courses that were at least one-day- 
long. Those courses were mainly doft skills” and statistics. In total, 330 trainees were invited to complete the 
instrument, and 182 complied (a response rate of 55 per cent). Results of the inventory were analyzed and fed-back 
to the organization during late-1998 and early-1999. Efforts to act on the results of the LTSI are currently being 
developed and implemented throughout the organization. 

Reflection on the Partnership 

In 1983, Schon introduced the notion of an epistemology of practice manifested through reflection- in- 
action. Reflection-in-action is based on the premise that knowing is tacit and demands critical analysis and 
reflection to turn thinking back into action. Usher and Bryant (1987) describe how the professional employs 
reflection to: 

...bring to the surface the implicit and tacit knowledge in the action which is integrated with that 
action and its outcome. The process involves an integral relationship between understanding 
(thinking), action (doing), and change. It is transactive in that an attempt to understand a situation 
through action leads to changes in the situation which themselves generate new understandings 
and renewed actions... . The metaphorical nature of the understanding enables problematic 
situations to be coped with by relating them to experience.. . As a result, new knowledge is being 
generated, (p. 205) 

The purpose of this study was simply to share practice-derived knowledge that was gained through the course of this 
research partnership by these two research partners. During the past two years, the authors have engaged in 
reflective analysis (Boxer, 1985) about the partnership itself parallel to the LTSI project. Data was gathered through 
observation, dialoguing with each other (typically followed-up by written notes), exchanging e-mails, and each 
research partner keeping periodic personal journals. The researchers actively discussed, framed, and interpreted 
activities and results associated with the project and the partnership. These discussions plus any existing notes and 
documents were then analyzed to extract key themes that are detailed below. 

The two-fold aim of our reflection-in-action was to (1) improve the LTSI research and the research 
partnership itself throughout the course of this still evolving partnership as well as to (2) more consciously analyze 
and document the partnership and the lessons that were being learned about conducting research in a practical 
setting. Schon (1983) explains that reflection-in-action is bounded by the ‘hction present, the zone of time in which 
action can still make a difference in the situation” (p. 62). Much of what is shared in the next section of this paper 
did indeed impact the LTSI project directly as well as the nature of the research partnership at the time. Other 
lessons shared here bear witness to fresher learnings that we hope will impact our future actions as we continue to 
push the deflective envelope” and the research agenda within the host organization. 
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Results 



The reflection and analysis of the two-year partnership identified three broad themes of learning points. These were: 
(1) involvement and understanding within the organization (dealing with stakeholder groups, education, language, 
and champions); (2) designing research for practice (linking to existing systems and strategies; and strategies for 
marketing, dissemination, and implementation); and (3) partnering for success (partnership objectives, relations, 
opportunities, and challenges). These are all explored in the following sections. 

Involvement and Understanding within the Organization 

This research project highlighted the importance of key stakeholder ownership, their understanding of the research 
process, and their need for assistance when interpreting and using the research results. 

Stakeholder groups. There were several key stakeholder groups for the LTSI project, other than the two 
researchers. Within the HRD department were trainers who would administer the instrument, HRD managers 
responsible for funding the project, and administrators responsible for data input. Organizationally other 
stakeholders included employees who would complete the instrument, training liaison officers who advise 
employees on training issues, and the organization^ Human Resources committee who had the power to action 
changes identified by the research. 

The project initially lacked an explicit strategy for managing the research-stakeholder interface and for 
developing a sense of ownership among all stakeholder groups. As a result, not all stakeholders received full and 
timely information about the research project and its progress which ultimately caused some resistance from 
stakeholders. For example, some of the trainers felt the project was an unnecessary addition to the HRD function^ 
workload, some work areas outside of HRD showed little interest in the research and the findings, and there were 
doubts about whether the project would be funded beyond the initial research period. These could have been 
lessened had the affected stakeholders felt a greater sense of ownership and been strategically involved from the 
outset. 

One method that should have been implemented much earlier in the process was to have enlisted several 
members of these various groups to act as champions and active participators. Although they were not all identified 
during the project, reflection has led us to believe that it would have been immensely useful to enlist a champion at 
the Board level to raise awareness of the research process and findings, a champion in the organization^ HR 
committee to foster the implementation of the findings, champions in each of the main work areas to encourage 
employee participation, and champions in the form of key trainers to encourage participation of training participants. 

Employees. In this organization, employees were viewed as a key stakeholder group as they would, after 
all, be invited to spend twenty minutes completing an instrument when attending a training course. Three methods 
were used to increase their sense of ownership: (1) the significance of the research was described to employees in a 
covering letter included with the instrument; (2) trainers gave a short verbal explanation of the research before 
handing out the instrument; and (3) a short article about the research was published in the monthly employee 
magazine. All three methods were delivered in commonsense and practical language used by employees rather than 
the language of research and academia. The hope was that the response rate would be bolstered by employees 
understanding of the reasoning behind the research and how the results would be used to benefit them and the 
organization. 

Even after this effort, the response rate was only 55 per cent. Informal surveys of non-respondents 
identified several reasons. Most notable was that employees did not perceive much benefit for investing the twenty 
minutes necessary to complete the instrument. Also, some respondents only completed part of the instrument 
because of what they viewed as ‘the repetitious nature” of the items. Others reported that they could not see the link 
between the items and the improvement of learning transfer. 

Language barriers. Language issues emerged in two distinct places during the project. First, items on the 
generic instrument had been written for use in a cross-section of organizations, but the generic language tended not 
to match that used commonly within this organization. Before administering the instrument, the researchers 
undertook an analysis of the questionnaire and changed references to better fit with the organization while also 
ensuring the validity of the instrument. Even with these types of edits, there were items and words that were 
difficult for some participants to understand. 

Another place that a language issue arose was when delivering results to key stakeholder groups. Wording 
of factor descriptions was considered by many in the organization to be ‘Academic” rather than ‘)?lain English”. 
This created a barrier to understanding the results. Here again, to reduce these effects, the researchers translated the 
results into the language of the organization, and most documents containing summaries of the research results 
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contained such translations. In both of these instances, the researchers had not anticipated such difficulty with the 
language and had deemed the documents as relatively consumable. It was a challenge for both the academic and the 
practioner to have the organization continuously ask for more practical language and to try to fulfill these requests 
while also being clear and accurate and maintaining scholarly integrity. 

Understanding research. This project drove home the importance of viewing the research process as 
educative within the host organization. Stakeholders in the organization required education on the research process 
and, most importantly, on accurately interpreting the results. The desire for practical and actionable results led to 
particular difficulty in three areas. First, was the tendency to overgeneralize given the sample size and strategy that 
had been implemented. Towards the end of the research, for instance, an interest emerged in comparing results 
between various courses, however the sample size for many individual courses was simply too small to produce 
reliable results at such a detailed level. It was also difficult to continuously remind stakeholders of the sample size 
in comparison with the population and to focus them on using the results effectively. 

A second area that demanded extensive education was around the distinction between items and factors. 
Stakeholders tended to want to interpret results at the item-level rather than the factor-level, which violates the 
statistical intent of factors and can lead to ineffective action within the organization. 

Finally, there was a need to educate stakeholders about the value of research and to help them gain patience 
for the process. In an organization experiencing much change, there was little patience, for instance, for what the 
researchers considered to be a thorough reporting of results. When stakeholders were provided with what the 
researchers considered to be succinct and thorough 5-10 page summaries of the results (derived from much longer 
reports), stakeholders reacted negatively and requested one-page summaries that were more like other business 
reports that circulate in the organization. Here again the researchers took steps to make the summaries shorter and 
more consumable, while also educating stakeholders to help them see the value of a longer description and its 
accompanying valuable detail. 

Designing Research for Practice 

Design of the research project is a critical component of its success. When researching in a practice-setting it is 
important that special considerations be factored into the design. Four critical strategic issues in this vein were 
identified through this project: (1) the importance of linking the research with existing strategies, (2) the need to 
support the organization in designing implementation strategies, and (3) the need for a systematic strategy for 
disseminating research findings. 

Links to existing strategies. The partnership research experience highlighted the benefits of building the 
research into existing strategies and systems. The LTSI was viewed in the organization as a tangential research 
project. Consequently it operated independently of existing systems, although its results were interpreted in light of 
outputs from existing systems and its findings were used to adapt existing systems. 

This perceived independence negatively influenced stakeholders’ perceptions. For example, some key 
stakeholder groups only began showing an interest in the research after being presented with the first results which 
illustrated the potential of using the LTSI. This then created interest at a senior management level, and resulted in 
additional funding for an expansion of the LTSI program. One way of linking it into existing strategies would have 
been to position the research from the outset as helping to explain the differences between the organization^ 
existing evaluation results at the learning and behavioral levels, which was an organizational problem that was 
already gaining some critical attention. This was not done and the momentum that this could have contributed could 
simply not be recovered. 

Finally, the LTSI instrument was not presented to course participants and other organizational parties as a 
part of the existing evaluation system already in place at the organization. It was marketed as a research project and 
consequently viewed as separate from the evaluation systems already in place. This was a particular challenge 
because part of the reason that the LTSI was set-up to participants this way was due to procedures around informed 
consent and wanting to ensure that LTSI participation remained voluntary. If it had been able to be done differently, 
it may have been easier to work it more seamlessly into existing systems. 

Implementation of research findings. The current project also emphasized the need for organizations to 
have a timetable and systematic strategy for implementing research findings. In this project, the results were made 
available to the HRD function in November 1998, however actions to address identified problem areas were only 
beginning to be planned in Spring 1999. Barriers to earlier implementation included the lack of a supporting 
champion in key committees within the organization and a view within stakeholder groups that the research results 
were of long-term importance rather than requiring immediate attention. Not having a solid plan to act on research 
results only reinforced stakeholder perception that research was an ‘fextra” thing to do. 
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Another barrier that was uncovered during the time that researchers delivered results to the organization 
was a perception that the research, and thus addressing the findings of the research, were the responsibility of the 
HRD function rather than of key people/functions in the organization. Stakeholders perceived HRD ownership for a 
combination of several reasons including: poor marketing by HRD of the benefits of the research to the 
organization, HRD initiation of the research without more intense subsequent development and involvement of 
stakeholders, and HRD sole-funding of the research. To a lesser extent, the perceptions of ownership by some 
managers reflected a deeper assumption that employee training and development are the responsibility of HRD 
(rather than of those managers). 

Dissemination. The research experience also highlighted the importance of the organization needing a 
systematic strategy for disseminating research results. Different stakeholder groups have varied needs for detail, 
timing, and format. In this project, it became clear that one format could not satisfy every stakeholder group — some 
formats provided too little information and others too much information. Different dissemination methods were 
therefore adopted. These included presentations to managers, materials for training courses, white papers for HR 
committees, and Powerpoint presentations for trainers. Although this approach increased stakeholder access to 
research information, the design and dissemination of the various formats was labor-intensive and time-consuming, 
especially for the internal researcher. The critical strategic disadvantage however, was that the full details of the 
research findings were primarily made available only within HRD (although were available on request to others). 
The main Experts” on the research findings were HRD employees, thus supporting the perception that this research 
was HRD-owned rather than owned by managers and employees. 

Marketing. The marketing of the research process has been a recurring theme throughout this paper, and 
merits recognition as a powerful lesson learned. Marketing is absolutely critical to the success of a research project 
inside an organization. Along with learning to conceive of the research process as educative, the researchers also 
learned how important it is to effectively market the project. It became clear that these researchers had much to 
learn about getting the message out there in an effective, engaging way. Some of the things that were done to 
address this included using a variety of internal mechanisms (e.g employee magazine), adapting formats and level of 
detail presented to various groups, and trying to keep the research project visible. 

Partnering for Success 

Reflecting on the two years of research collaboration, a few main issues emerged as important in influencing the 
quality and effectiveness of the partnership. 

Mutual roles and goals. The two partnering researchers were aware and explicit from the outset that the 
research was seeking to address two sets of objectives. The first set came from the organization and included: 

• that outputs be sufficiently robust for practical application; 

• that the project be kept within financial resource limits; 

• not placing an undue burden on HRD trainers and administrators; 

• not requiring course participants to spend too long completing the instrument; 

• working within organization timeframes; 

• moving the research through its various stages only when necessary financial support was 
secured. 

The second set of objectives came from the desire to contribute to the validation of the LTSI instrument. This 
required academic rigor on data collection methods, sufficient sample size, using the instrument as originally 
designed rather than too adapted for the organization, discipline in administration of the instrument, and data 
management. 

In this project the partners were fortunate that, for the most part, the academic^ goals and those of the 
practioner were quite compatible. As Jacobs (1999) advises, it was important to define and discuss these at the 
beginning of the partnership and ensure that the goals of each researcher are not mutually exclusive. This was a 
relatively easy task in this project because the academic fc goals were rather minimal in that the research activity was 
simply the administration of the instrument. Beyond that, much was negotiable to easily accommodate the 
organization. This flexibility also allowed the researchers to pursue the secondary and tertiary research questions 
depending on various levels of organizational commitment. 

Another thing that emerged as a powerful theme during the analysis of this research partnership was how 
the academic and the practioner involved in this project both positioned themselves as researchers. A review of the 
literature has shown that it is not uncommon in case studies describing research/practice partnership to clearly 
differentiate between the ‘^researcher” and the practioner”. This was not done during this project, and it 
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fundamentally shaped the project and the partnership. Rather, each partner involved here was viewed as an HRD 
professional who offered a specific set of competencies, expertise, and resources and whofc primary goal was to 
research and then contribute new knowledge to the field of HRD. The practitioner of the team is an active and 
developing scholar with a clear commitment to research. The academic of the team was , of course, highly motivated 
to research, while also striving to be an excellent practioner. So, when necessary, each of the partners worked hard 
to put on the ‘Researcher” hat or the ‘practioner” hat. These roles and their responsibilities were shared as much as 
possible rather than delineating specific things to a specific role because one person happened to work in academia 
and the other in an organization. Due to this, mutual goals included wanting to ensure robust and actionable 
findings for the organization and, for instance, the goal of publishing article(s) together related to this project. 

Viewing the roles as shared in this way also emphasized the importance of each partner educating the other 
throughout the process. From the academic^ perspective, that included providing key (but certainly not all) 
information on the research process, research design detail (such as sample sizes), the interpretation of results (for 
example, interpreting item-level and factor-level results), and on the publication of results. From the practitioner^ 
perspective, that included providing important (but once again not all) knowledge about the organization, on the 
stakeholder groups, advice on issues of timing and how much to ‘£ush”, and on how best to communicate aspects of 
the research project in the organization. All of this was negotiated with both partners holding both sets of goals as 
equally important and by seeing each role as central to achieving those goals. 

Building and sustaining the partnership. The experience highlighted the importance of three factors in the 
building and sustaining of a successful research partnership. First is simply the issue of time. Relationships take 
time to develop, and time is necessary for the partners to learn about each otherfc values and beliefs, working styles, 
objectives, and goals. Such an understanding emerges over a period of working together, and effort is needed to 
maintain the relationship once built. 

Second, frequency of contact was very important to these researchers during this project. Building the 
partnering relationship required regular contact, both in the planning of the research and during the subsequent 
stages. Regular contact allowed both researchers to explore each otherk experience of the research in greater depth 
than would be possible with occasional dialogue. In this project, it was noticeable that the momentum was greatest 
around the time of a planned contact between the two researchers and tended to slacken if the gap between contacts 
was too great. 

Finally, trust is a central tenant of this partnership and the researchers invested the energy to ensure it. In 
the case of this research, trust was developed through frequent open dialogue about the research process itself. This 
included making plans, airing any concerns that emerged, and diligently working to clear up any confusions. 

Working " virtual ” This partnership research was completed across a four thousand mile divide between 
the academic in the US and the practitioner in London. This introduced a few special challenges. For instance, the 
project was rarely discussed face-to-face, so we committed to regular telephone calls (arranged around the six hour 
time difference between MN and the UK). Much of the communication between the researchers was via e-mail, 
which led to delays in responses and a need to anticipate activities/events. 

Another big challenge was that the researcher was unable to visit the organization or meet any of the 
stakeholder. She thus had to rely on the practitioner researcher for all contextual information and, from the 
organization^ perspective, key stakeholders never met the academic researcher, which meant their relationship with 
the research was exclusively with the internal partner. The latter consequence could have influenced stakeholder 
perceptions of the research. None of these problems were sufficiently large to cause major problems, but all added 
(in one way or another) to the challenges of partnership research. 



Implications 

Merriam (1986) once stated that ‘Research and practice... can each be enriched through closer contact with each 
other”. Closer contact, though, brings with it inherent challenges and these must be explored. This reflective piece 
from two researchers is not any kind of model for effective partnerships — but it does document some of the lessons 
learned in the course of administering and acting on results of the LTSI at this large public organization. These 
implications for other researchers to consider as they design and implement their research partnership are offered in 
Figure 1. 
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Figure 1 . Kev Lessons for a Successful HRD Research Partnership, 



INVOLVEMENT AND UNDERSTANDING IN THE ORGANIZATION 


• 


Identify and enlist the support of main stakeholder groups. Pay particular attention to budget holders, 
stakeholders within HRD, key decision makers outside of HRD, and employees who will be invited to 




participate. 


• 


Create a strategy for developing champions for the research project within HRD, in key committees, and 
within work areas. Involve them as much as possible in the design and implementation of the process. 


• 


Reach out to employees who participate in the research and help them to see the value of their participation. 


• 


Ike the language of employees - avoid the language of research and academia where that creates a barrier. 


• 


View research in applied settings as an educative one. Be ready to educate, educate, educate! Stakeholders 
and champions need education about many aspects of the process (objectives, timeline, resources) and must 
be helped to understand the process and its value. 


• 


Educate key stakeholders and champions about research design in advance of research implementation so 
that they understand the implications of design issues. ( e.g. sample size, generalizability, etc. . . ) 




DESIGNING RESEARCH FOR PRACTICE 


• 


Where possible, link the research project with existing organization strategies and systems. 


• 


Develop an implementation strategy with the involvement of key champions to increase support for early 
consideration of findings. 


• 


Be clear from the outset who/what function must be primed to act on the findings of the research project. 
Involve them in the research design and implementation. 


• 


Begin planning to take action on the research findings during initial design of the research project. 


• 


Develop a systematic dissemination strategy that provides each stakeholder group with the right level of 
detail in a language they will understand, and at the right time. 


• 


Develop a comprehensive marketing strategy based on the needs of various stakeholder groups and utilizing 
different delivery formats and varied levels of detail (based on those needs). 

- Access internal communication avenues to market the research (e.g. employee magazines). 

- Keep the research visible throughout the research process, particularly during data collection and analysis 
stages, when stakeholder groups may otherwise see less in terms of progress. 

- Market the research in the language of stakeholder groups, avoiding academic language. 

- Emphasize the practical benefits of the research to stakeholder groups. 



PARTNERSHIP 

• Explicitly identify each parties objectives at the outset and design the research to best address all mutually- 
agreed upon goals. Within those objectives, include the organization^ financial and non-financial resource 
constraints, organizational timeframes, and the desired utility of results beyond the immediate organization. 

• View both (all) partners as researchers, rather than ‘Researcher” and ‘))ractioner”. Share the researcher role 

and responsibilities. 

• Work to build and sustain an effective trusting relationship over a long period of time, based on frequent 

contact and an awareness of each others* * motivations and goals for the research and the relationship. 

• Take advantage of the development opportunities of working with each others, accepting that each brings 

different knowledge, expertise, and experiences to the partnership. 

• Anticipate and address challenges presented by partnering long-distance (such as logistical problems like 

time differences, lack of face-to-face contact between partners, and lack of contact between the academic 
partner and key stakeholders. 



Limitations and Conclusion 

The findings reported here are not to be overgeneralized. Once again, the purpose of this paper was simply to share 
practice-driven knowledge that these two researchers have gained and applied during the course of this evolving 
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research partnership. These findings are specific to this situation and these researchers, and the lessons that have 
been extracted are necessarily limited by the state of development of each of these researchers. 

However, it is hoped that, like other pieces of this nature, the results will help other HRD professionals who 
are working together as research partners ‘to engage in reflective observation and dialogue about their own efforts” 
(Kehrhahn & Verrilli, 1999) and encourage them to share their learnings with others so that we may all continue to 
develop additional specific expertise around research partnerships. 
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